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ABSTRACT 

A study examined in detail the initial encoding of 
worked-out examples of mechanics problems by "good** and **poor** 
students, and their subsequent reliance on examples during problem 
solving. The subjects, three males and five females, were selected 
from responses to a university campus advertisement. Six of them were 
working towards bachelor's degrees with varying majors, two of the 
eight students had additional post-graduate training in psychology 
and none had a college physics course, although seven of the eight 
had taken physics in high school. Students with a range of 
grade-point averages were chosen so that learner differences could be 
examined. Subjects were given instruction in physics and required to 
demonstrate mastery of basic physical principles. Talk-aloud 
protocols were employed to examine the way the subjects learned and 
understood the examples, and then used their understanding to solve 
problems. The subjects were split into two groups post hoc using a 
median split on their problem solving success. Results indicated that 
**good** students tend to study examples and exercises by explaining 
and providing justifications for each action and relate their 
explanations to the principles and concepts in the text. **Poor** 
students do not often explain the examples or exercr.ses to 
themselves, and when they do, their explanations do not seem to 
connect their understanding with the principles and concepts in the 
text. The results provide, at a gross level, empirical evidence to 
support existing artificial intelligence models of explanation-based 
generalizations. (Seven figures and seven tables of data are included 
and 27 references are appended.) (RS) 
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Abstract 

The present paper analyzes in detail (talk-aloud protocols) "Good" and "Poor* 
students* initial encoding of worked-out examples of mechanics problems, and their 
subsequent reliance on examples during problem solving. We find that "Good* students 
lear-n with understanding: they generate many explanations which refine and expand the 
conditions for the action parts of the example solutions* and relate these actions to 
principles in the text. These self-explanations are guided by accurate monitoring of their 
comprehension failures and successes. Such learning results in an example-independent 
knowledge and in a better understanding of the principles presented in the text. ■Poor" 
students do not generate sufficient self-explanations* inaccurately monitor their learning 
and subsequently rely heavily on examples. The results are discussed relating these 
psychological findings to existing AI models of explanation-based generalizations. 
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SELF-EXPLANATIONS: HOW STUDENTS STUDY AND 
USE EXAMPLES IN LEARNING TO SOLVE PROBLEMS 

Learning Is a constructive process in which the student converts the words, and 

examples from the teacher or the text, into usable skills, such as solving problems. This 

process of conversion is essentially a form of constructive selfrinstruction (Simon, 1S79). 

Although the research on the quality of good teaching (such as those which attempt to 

identify the characteristics of a good Socratic tutor, Collins & Stevens, 1982), as well as 

research on the quality of a good text (such as those that manipulate the quality of 

elaborations, Reder & Anderson, 1980) may be informative, ultimately, learning rests on 

the learning skills that the students themselves bring to bear as they learn. The goal of 

this research Is to understand the students' contribution to learning. To accomplish this 

goal, our approach is to study individual differences as the students learn. 

The majority of problem solving research in the literature assumes that the 
declarative Knowledge is first encoded from the text or from the teacher's words, then 
proceduralized into a skill. Most of the literature has concentrated on the conversion of 
already encoded knowledge into smooth, fast, skillful problem solving. This conversion 
process dominates, for example, Anderson's theory of skill acquisition (Anderson, 1987). 
In that theory, the process of conversion is achieved by using general weak methods 
which can convert declarative knowledge into domain specific procedures via the 
mechanism of compilation. Thus, in Anderson's theory, it is assumed that the effortful 
process lies in the conversion of the declarative knowledge Into the procedural 
knowledge, whereas the encoding of the declarative knowledge is taken to be a 
straightforward storing: 

In unanalyzed form our experiences in any domain, including Instruction (If it 
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Is available), models of correct behavior (worked-out examples), successes and 
failures of our attempts, and so on... This means that we can easily get 
relevant knowledge Into our system.... (p. 206). 

We concur with the common assumption that learning a skill can be viewed as 
encoding of instruction followed by procedurallzatlon of some kind. However, our 
research focuses* on the encoding of instruction, because a leading conjecture is that 
individual difference is due to the differences in the encoded representation rather than 
differences In conversion of the encoded instructions to skill. Anderson (1987) also 
alluded to this difference in the encoded representation as a source of individual 
differences when he suggested that: "weak problem-solving methods like analogy can be 
much more effective if they operate on a rich representation of the knowledge...* (p. 

206). 

Our work focused on the knowledge that students acquire as they read from a text, 
prior to their attempts to apply that knowledge In the form of a procedural skill such as 
problem solving. We predict that learner differences arise primarily from the 
representation that the student has acquired. Thus, we believe that individual 
differences do not arise solely from the experiences the learner Is exposed to in the form 
of Instruction or text. F?ather, we conjecture that learner differences may arise from 
differences in the ways students understand and learn from text, and moreover, these 
learner differences subsequently effect the use of a particular skill (solving problems). 

Our ^"^^^rest In learner differences derives in part from an attempt to understand 
the acquisition of expertise. While it takes long periods of study and practice before one 



can- become an expert in any domain. It Is clear thht extensive practice and study Is a 
necessary but not sufficient criterion for becoming an expert. Thus, examining how 
students differ in the way they learn new materials may shed light on processes of skill 
acquisition that determine whether an individual will or will not achieve expertise. 

The part of the text that we focus on is the worked-out examples. This choice was 
based on theoretical, empirical, as well as instructional reasons. Theoretically, there is 
some controversy in the literature concerning how generalizations are induced from 
examples (see Dletterlch & MIchalskl. 1983, for a review of the AI literature, and 
Murphy & Medin, 1985, for a review of the psychological literature.) There are two 
views. A similarity-based approach claims that generalizations are developed by inducing 
a principle (or a set of common features) from multiple examples. Such a principle would 
embody the essential features shared by all the examples. On the other hand, an 
explanation-based approach to problem solving (Lewis, 1986; Mitchell, Keller & Kedar- 
Cabelli, 1986) claims that generalization can be obtainf* from a single or a few 
examples. Although many theories, including Anderson's ACT*, can only generalize 
from many examples, it is clear that students can often generalize from a single example 
(Elio & Anderson, 1983: Kieras & Bovalr, 1986). Providing evidence on how 
generalizations can emerge from learning a few examples would give credence to an 
explanation-based approach to learning. 

Empirically and instructionally, there Is a dilemma in the literature as well. More 
and more empirical evidence is emerging showing the importance of examples In 
learning. Reder, Charney and Morgan (1986) for example, found that the most effective 
manuals for Instructing students how to use a personal computer are those -A^hlch contain 



examples. LePevre and Dixon (1986) found that students actually prefer to use the 
example Information and l^ored the written Instruction when learning a procedural 
task. Plrolll and Anderson (1985) also found that 18 of their 19 novices rely ou analogies 
to examples in the early stages of learning to program recursion. Besides these 
laboratory findings. VanLehn (1986) also provided Indirect evidence that examples are 
Important In regular classroom learning. He found that 85% of the systematic errors In 
arithmetic, collected from several thousand students could be explained as deriving from 
some type of example-driven learning process. 

On the other hand, although both students and Instructional materials rely heavily 
on worked-out examples as an Instrument for learning, the empirical work which directly 
examined the role of example solutions on problem solving found that students who have 
studied examples often cannot solve problems that require a very slight deviation from 
the example solution (Eylon & Helfman, 1982; Reed, Dempster, & Ettlnger, 1985; Sweller 
& Cooper, 1985). The discrepancy between students* better performance from text 
materials that contain examples and the students* failure to generalize from examples, 
may be caused by the degree to which students understand the examples provided. 
Generally, in the empirical studies cited, no assessment is made about how well students 
understood the examples. As Plrolll and Anderson (1985) noted, although most of the 
students wrote new programs by analogy to example programs, the success depended on 
how well the students understood why the examples worked. Our work, in the domain 
of mechanics, will shed light on the degree to which students understand an example in 
relation to their ability to solve problems. 

The method we use to study how stud.^nts learn and understand an example is via 



the explanations they g?ve v'hlle stiadylng It. Such self-explanations a?8 Important, we 
propose, largely because examples typically contaiii a sequence of actions, without much 
explication of the rationale underlying the sequence of actions. Simon (1979). for 
example, noted that: 

Generally speaking, textbooks are much more explicit in enunciating; the laws 
of mathematics or of nature than in saying anything about when these laws 
may be useful in solving problems. The actions of the productions needed to 
solve problems in the domain of the textbooks are laid out systematically, but 
they are not securely connected with the conditions that should evoke them (p. 
02). 

We can provide a concrete example of the inadequacy of worked-out examples by 
taking one example from the fifth chapter of the Halliday and Resn'^ .c text (1981). as 
shown in Figure l. We can see the lack of specification of the explicit conditions under 
which the actions should be executed. 

For example, it is not clear in Statement 2 of this example-exercise why one should 
■consider the knot at the junction of the three strings to be the body." This is a critical 
piece of information because it implies that at this location (as opposed to the block), the 
sum of the forces is zero. Such lack of specificaiion Oi the explicit conditions for actions 
occurs throughout the example. In Statement 6. how does the student know that F.. Fq. 
and F^ are all the forces acting on the body, and that there are no others? Statement 7 
Is essentially a restatement of Newton's First Law.^ but it requires chaining several 
Inferences together, and translating them into an equation (for example, because the 
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Pi9ur« 5*6 Exampit 5. (a) A block of wtighi \\ $us)tndtd 
by itringi. (h) A /rtt*body diagram showing all the forces 
acting on the knot. The strings art asaumed to be weightless. 



I. Fiftire 5-a& shows ao object of weif hi W buDf by musless strings. 

2« Consider the koot \x tbe juQctios of tbe tbree striop to be "the body*. 

3. The'body remaios at rest under the action of tbe tbree forces sbown 

Fif . s.e. 

4. Suppose we are f iven tbe magnitude of one of tbese forces. 

5. How can we find tbe magnitude of tbe otber forces? 

F^, Fq, and F^ are a// tbe forces acttnf on tbe body. 

7* Since tbe body is unaccelerated, F . + F^ + F- » 0 

A o C 

Cboosinf tbe x* and y*axes as sbown, we can write tbts vector equation 
as tbree scaiar tquation;: 

tl. using Eq. 5.2. Tbe tbird scaiar.equation for tbe z*axts is simply: 

13. Tbat is, tbe vectors all He in tbe x-y plane so tbat tbey bave no t 

components. 

14. From tbe Hgure we see tbat 

tS- F^ — .F^ cos 30 — -0.888F ^, 
e 

^Ay"*^A"°^^ — O.SOOF^. 
t7. and 

a 

Fgy — Fg S'm 45 — 0.707 Fg. 



Figure 1: A strings example, taken directly from Halliday and Resnick 

(1981) 
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body Is at rest, there are no external forces, therefore the sum of the forces on the body 
must equal to zero)* Statement 8 is totally unexplained; why are the axes chosen as 
such? It is clear that the solution steps within an example are not explicit about the 
conditions under which the actions apply. 

In order to learn with understanding, a student needs to overcome the 
incompleteness of an example by drawing conclusions and making inferences from the 
presented information (Wickelgren, 1974)* To do so, a student needs to provide 
explanations for why a particular action is taken* Only then wiL the student be able to 
apply the acquired procedure to non^isomorphic problems that do not match exactly the 
conditions of the example solution. Thus, we suggest that a good student "understands" 
an example solution and will succeed in generalizing because he/she makes a conscious 
effort to explain and extrapolate the principle, and to ascertain the conditions of 
application of the solution steps beyond what is explicitly stated. Consequently, we ask 
in this paper how students can come to understand an example. 

It is difficult to define what ■understanding* means in the context of learning frpni 
examples. Operationally, our study was designed to assess understanding with three, 
measures: solution of isomorphic problems, solution of far transfer problems, and 
elaborations generated during studying examples. The weakest method to detect 
understanding is to observe how successfully students solve very similar problems, since 
very similar problems can often be solved by a simple syntactic mapping of the example 
procedure to the to-be-solved problem. Another method is seeing if students can 
successfully use the principle involved in the example in a different and complex 
problem, (far transfer), because such problems prevent students from being able to solve 
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ihem via a syntactic mapplne. However* to a certain extent, studying far transfer reveals 
only that understanding exists and allows one to see what conditions facilitate It. 

The method which permits the most direct assessment of understanding of an 
example Is to examine the explicit elaborations that students provide while studying It 
Elaborating Is a mechanism of study that allows students to explain* Infer* and explicate 
the conditions and consequences of each procedural step In the example. We postulate 
that elaborations can reveal students* understandmg by showing whether or not they 
know (a) the conditions of application of the actions, (b) the consequences of actions, (c) 
the relationship of actions to goals, and (/J) the relationship of goals and actions to 
natural laws and to other principles. This paper focuses primarily on the elaboration 
results. 

In the actual procedure used, we invited students to explain to themselves what 
they understand after every line of a worked-out example. Often they made no 
comments at all. However, when they did generate a comment, this method allowed us 
to yoke the explanation to the statement ;line in the example so that we could interpret 
their protocols more easily. 

Method 
Subjects 

Eight students (3 males and 5 females) were selected from responses to a campus 
Advertisement, Six of them were working towards bachelor's degrees with varying 
majors. Two of the 8 students had additional post-graduate training. In psychology. None 
of the students had a coUe physics course, although all of them (except one) had takon 
high school physics, with differential performance (reported grade) In that course. We 
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(ntentlonally chose students with a ranee of abilities in terms of erade-point average so 
that we could examine learner differences. Students were paid for their participation. 

Procedure agd Materials - 
In the piresent paper we focus on how students study three worked-oat examples of 
problems dealing with the application of Newton's laws of motion and on how this initial 
learning relates to their subsequent problem solving* However, since the relevant 
chapter demands substantive background subject matter, the study of examples and the 
problem solving tasks are embedded within a loneltudinal study in which students in our 
laboratory studied Newtonian Mechanics, Students learned the new material in a way 
they would normally do when studying on their own, in terms of the amount of time 
devoted' to studying, the rate of self-pacing, and the use of studying habits, such as 
highlighting significant parts of the text or rereading. The laboratory learning differed 
from the way students would typically learn on their own in that all the learning took 
place in the laboratory, and that students gave talk-aloud protocols while studying 
examples and solving problems. The students spent between 8 to 29 hours to complete 
the study, spread over several weeks. Figure 2 presents a diagram sumniarizlng the 
experimental procedure for the whole study. Basically, the study consisted of 2 major 
phases: knowledge acquisition and problem solving. 
Knowledge Acquisition 

During the^ first .part of the knowledge-acquisition phase, siibjcts studied the 
necessary background subject-matter, covering the topics of measurement, vectors, and 
.mptlpn in one dimension. These materials are covered In the first three chapters of 
HallWay & Resnlck; The fourth/chapter on Motion Jn a Plane- was; e^^^^ 
did hot have direct t^earing on learning Chapter 5v tiie target chapter on Particle 
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Pretests 



Knowledge Aquisition 



Background Knowledge 









1 












Chapter 1 






Chapter 2 






Chapter 3 











Target Knowledge 




Chapter 5 



Text 

Study 



Examples 
iStudy-T) 



Problem Solving 



Isomorphic Problems 

String Incline Pulley 



Simiiiar 

Changed 
Gravity 

Changed 
Force Dir. 




Chapter 5 Problems 



(7 problems) 



Problem Anticipation 



Procedural Components 



Posttests 



Figure 2: A diagram depicting the design of the study 
Dynamics. Decisions of this kind, as well as designing of questions and problems, were 
made after consultations with physicists. 
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For each of the three background chapters, students read through and studied the 
chapter. They were requested to record on a separate sheet of paper any questions 
which arose during their study. Each chapter also contained a note to stop and get the 
experimenter in order to give a verbal protocol of one worked-out example. This protocol 
was meant mainly as practice in giving protocols. When the students believed they were 
ready to be tested on the material in the chapter, they notified, the experimenter, and 
were then required to produce correct answers to a set of Declarative, Qualitative and 
Quantitative questions, given in that order. Declarative questions were designed to 
assess the recall of critical facts from each chapter, e.g., *What is the difference between 
a scalar and a vector?" Some of these questions came from the textbook and some were 
designed by the experimenters. The Qualitative questions were designed to assess 
reasoning and inferences about the concepts in each of the chapters without reference to 
quantities, e.g., "Can an object have an eastward velocity while experiencing a westward 
acceleration?" The majority of these problems were taken from- the textbook, several 
were generated by the experimenters. The Quantitative questions assessed procedural 
skills for quantitative problem solving, e.g., "Tv/c bodies begin free fall from rest at the 
same height, 1.0 sec apart. How long after the first body begins to fall will the two 
bodies be 10 meters apart?" All of the Quantitative problems came from those presented 
in the text at the end of the chapters. 

After solving each problem set, students returned the problems to the experimenter 
for grading. If no errors were made, they proceeded to the next set of questions. If an 
error was made, the questions were returned for correction. The incorrect answers were 
Identified and students were given references to those sections of the text which 
addressed i::ese questions. For incorrect Quantitative questions the correct final answer 
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was also provided, analogous to the common practice ot iooklng up the answer In the 
back of the book* Students then attempted to correct the answers and resubmitted them 
for grading. If there were still errors on the second try, students were provided with a 
worked-out solution for the incorrect answers. Students then studied the worked out 
solution, and after it had been removed, were asked to change their answers and/or to 
reproduce the correct solution from memory. If after these two tries the answer was still 
incorrect, V/c experimenter explained the worked*out solution and the student had to 
reproduce it. Thus, the first three chapters, consisting of the background subject matter, 
were studied until students could correctly solve a set of Declarative, Qualitative and 
Quantitative problems relevant to each chapter. 

During the second part of the knowledge acquisition phase, students studied the 
target chapter on particle dynamics (Chapter 5 of Halliday & Resnick). Studying the 
target chapter proceeded exactly as for the other chapters, except that students were 
required to answer correctly the relevant Declarative and Qualitative questions before 
studying the three worked-out examples. This was done to assure that the students bad 
acquired the relevant declarative knowledge from the text. 

The study of examples was the major focus of the knowledge acquisition phase; 
students studied three worked-out examples, taken directly from the text. {See Figure i 
for. an example of one of the worked-out solutions.) Each example solution represented a 
■type" of problem. There were three types: a strings, an inclined plane, and a pulley 
problem. (See Figure 3.) 

Students talked out-loud while studying the examples, and their protocols were taped. 
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Problem solving 

Ddrlng the problem solving phase, students solved two main sets of problems. The 
first vCt consisted of 12 'isomorphic problems", with four problems corresponding to 
each "type* of example (see the shaded areas in the problem solving phase of Figure 2). 
These problems were specially designed by us so that they varied in their degrees of 
similarity to the worked-out examples studied in the previous phase. Figure 4 shows one 
set of Isomorphs corresponding to the "strings" example. 

The second set consisted of seven "chapter" problems, which were problems taken 
directly from the end of the target chapter. In terms of our criteria for isomorphism to 
example problems, these problems can be considered "far transfer" problems. Students 
solved both sets of problems while giving talk-aloud protocols, and no feedback was 
provided. Only the problem solving protocols of the first set of three Isomorph problems 
will be analyzed in detail for this paper. 

Results 

As indicated, this paper v»111 report on the results of the elaboration protocols while 
students study the example solutions, presented in the target chapter, as well as how 
they use the examples in the problem solving protocols of the Isomorphic problems. 
Individual differences will be reported by contrasting the performance of "good" and 
"poor" students. These two groups were defined post hoc, using a median split on the 
problem solving successes of our students on the 12 isomorphic and the 7 chapter 
problems. In scoring the problems, arithmetic slips were not considered as errors and 
partial solutions were credited proportionally. All problems received the same weight, so 
that the maximum score for the isomorphic problems was 12 and the maximum score for 
the chapter problems was 7, (All scoring, identification and classification of protocols 
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A block is buKlns tmm three strlncL If the teuton In strlns 1 ts 18 N, wbu ts the mass 
of thi blockT ^ ^ . \ ^ . ^ ^ , ^ ^ - 



SImiUr 



Th* block Bictured ts htnclnc from 3 strlnts. The mass of the block ts 10 kg. If the 
rope A be? 



Changed Gravity 




m 



A balloon is bclDt held down by three masaless ropes. If tie balloon Is pulline up with a 
force of 300 N. what would the tension In rope A be? 




Changed Force Direction 



W777? 



77P77r777Z'777777777^777777? 



Three forces are holding a 80O kt block motionless on a frfctionless surface. If force A Is 
^ N. what would force C bv*? 



New Surface 




Figure 4: The set of four isomorphs to the strings example 
were performed by two Judges, with a mean Interrater reliability across all the analysis 
ranging between 86% and 94%.) 



There were four students in each group. The mean success of the Good students 
was 82% (96% for the isomorphic problems and 68% for the chapter problems). The 
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mean success of the Poor students was 46% (62% for the Isomorphic and 30% for the 

chapter problems). 

Learning from Examples: Elaboration Drotocols 

Amount of Elaborations Generated: Unit of Analyses. 

Our first analysis began with a very eross count of the amount of elaborations 
students provided while studying the example solutions. An elaboration Is a generic term 
we use to Indicate any statement made that is not a first reading of an example line, nor 
is It any conversation carried on with the experimenter that does not refer to the subject 
matter of the example, such as "Do you have n calculator that I can use?*, nor is it a 
response to the experimenter's requests to speak louder. Figure 5 is a transcript of one 
student's elaboration protocol. During the transcription, segmentation was made roughly 
at pause boundaries. We also assigned a line number to each line of the protocols. 
Counting each line that Is not a reading or experimenter's comment as a line, this 
example shows a total of 13 lines. Notice that because we are not using sentence 
boundary as a line boundary necessarily, a short comment such as "Okay' is counted as 
one line, since it is Inserted between different activities, such as reading. 



Table 1 shows the average number of lines generated per example by the Good and 
Poor students, as well as the average amount of time it took them to study an example. 
These differences are contrasted with the average number of protocol lines generated and 
the amount of time taken to solve an isomorphic problem. The Good students generated 
' a considerably greater number of elaboration lines than the Poor students (142 lines vs. 
21 lines, t(6) = 1.97, p < 0.05). These elaboration activities necessarily led the Good 
students to spend more time on each example as well (13 mln. vs. 7.4 mln., t(6) = 2.16, 
p < .05). 
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Read Line 6: 

(I) Elaboration: Monitoring 
' Read Line 1. 

Experimenter: 

(n) Elaboration: Monitoring 

(m) Elaboration: Explanation 

(IV) Elaboration: Explanation 

(V) ' Elaboration: Explanation 

(VI) Elaboration: Explanation 

(VII) Elaboration: Explanation 



Fa, Fbf and Fc are all the forces 
acting on the body, (pause) 

1) Okay. 

Since the ... (pause) 

(Okay, what are you thinking about?) 

2) rm trying to think where Forces Fb and Fa 

3) are going to get the thing* 

4) They*d just be the force, the rest mass of 

5) the thing holding it up would be the force* 

6) It could, well, actually it*d be the force of 
weight. 

7) Cause being upheld by..Jt*s the resistance 

8) to weight W. 

0) It would all be equal. 

10) The change In the uh, negative Y will the... 

11) The force of negative Y will be equal to 

12) the force in the positive Y. 

13) And they*ll all equal out. 



Figure 5: An example of a transcript of example-studying protocols (SlOl) 



We rule out the possibility that Good students are simply more articulate or fluent, 
thus producing more elaborations, since, in contrast, the ^umber of lines they generate 
while solving problems Is approximately the same as the Poor students (141 vs. 122 lines 
per problem, see Table 1. bottom). This suggests that the students generate as many 
lines of protocols as they deem necessary in order to learn the example. Likewise, the 
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Table 1 

Amount of Elaboration Generated While 
Studying Examples and SglTiug Problems 



GOOD POOR 

Example- No. of Lines 142 21 

Studying 

Protocols No. of Mln. 13 7 



No. of Statements 
• or Single Ideas 51 18 



* Isomorphic No. of Lines 141 122 

Problem 

Solving No. of Mln. 13.8 14.3 

Protocols 



Note. The numbers represent averages per student per example. 

fact that the Good students took more time studying each example solution than the 
Poor students (13 mln. vs. 7.4 mln,) also reflects their choice to spend more time on each 
example, rather than a tendency to dwell unnecessarily on the examples, since the 
amount of time they spent solving each isomorphic problem Is about the same as the 
time spent by the Poor students (13.8 mln. vs. 14.3 mln.). 
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The assumption we make, that elaborations are generated If some knowleige or 
Infersnce Is being processed or constructed In memory, is no differ cut than assumptions 
made with reference to other types of dependent measures. For example, the assumption 
underlying protocol analyses is that the problem solver is saying what he is thinking 
about or dumping the content of his/her working memory. Thus, longer protocols simply 
refer to a greater degree of procv^slng. Similarly, in research using eye movement 
protocols, the assumption Is that the student Is processing the location at which he/she is 
fixating (Just & Carpenter, 1976), Thus, longer fixations imply that the student is 
spending more time processing that location. Therefore, we view the greater amount of 
elaboration produced by the Good students as a natural consequence of wanting to 
understand the solution example better, rather than the possibility that they are more 
articulate and fluent. 

It might be claimed that the reason that Good students learned more Is Just that 
they spent more time studying. However, this is not a very deep explanation, but merely 
a restatement of the correlation. The important question is what the Good students did 
while they were studying. As will be seen shortly, there are such large qualitative 
differences In what students did while studying examples that the simple, shallow, 
explanatlon-that studying twice as long makes one learn twice as much-Is quite 
Implausible because It cannot explain the other differences In students^ learning bc'iavlor. 

However, although we claim that longer protocols do not necessarily Imply 
verbosity, using lines as a unit of analysis does not seem adequate for capturing what Is 
going on. That Is, an analysis at the level of lines Is too fine-grained to characterize the 
nature of these comments. Instead, many lines seem to refer to a single idea, and thus 
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"we further collapsed the lines into units expressing a sinele idea. For example, the 13 
protocol lines in Fie, 5 have been recoded Into 7 elaboration "statements* (I-VU)- Thus, 
basically the boundaries >vere placed on the basis of different Ideas, pauses, or different 
sort of activities including reading or experimenter's interjections. 

Using this sort of parsing, Ave see In Table l that Good students' protocols have 
been reduced from 142 lines to 51 statements and Poor students' protocols remain 
roughly the same (that is, 18 statements vs, 21 lines). Thus, in a sense we are penalizing 
the Good students for being articulate. What the data actually shows is that the Good 
students often elaborate quite a bit on a single idea either because they realize that they 
do not understand it or they want to provide a more complete explanation (see next 
sections), or the Ideas are just harder to explain succinctly, Lewis and Mack (1982) have 
also noticed that learners often spontaneously offer explanations of why things happened 
the way they did. Even \/ith this unit of analysis, the Good students elaborate 
significantly more frequently than the Poor students (51 vs, 18, t(6) = 1,98, p < ,05), 
Henceforth, all analyses will be based on the number of single ideas or statements. 
Kinds of Elaborations Generated. 

Elaboration statements can be classified into three types, (See Fig. 6 for a 
breakdown of all the classifications to be discussed,) 

An elaboration is considered to be an explanation If it says anything substantive about 
the physics discussed in the exo.mple statement. The following comment would be 
considered an explanation: 

'Ummm, this would make aenae, becauae since they're connected 
by a string that doesn '/ slretcL ' 
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Elaborations 




Justificition Diagramatic Others Ncwton*s Laws 




Refine/ Explicate/ Impose Give meaning 
expand infer a goal to quantitative 

conditions consequences expressions 



Figure 6: A classification scheme illustratine he decomposition of 

the analyses of the protocols. 

or 

'If the string's going to be stretched^ the earth !s going to be moved^ 
and the surface of the incline is going to be depressed. ' 



Other examples can be seen In Figure 5. 

An elaboration is considered a monitoring stateme^ it if It refers to states of 
comprehension. For example, remarks such as: 
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V can 8tt now how they did it' 
or 

'/ was having trouble with F-rngsinQ = 0' 

would* be considered moDitorlDg statements* 

A third category includes several other types of statementSt mostly paraphrasing, 
mathematical elaborations, and metastrategic statements. Paraphrasing would be 
comments that restate what the example ?ine said. For example, a student who read the 
example line 

Notice that the magnitude of T is always intermedijite 
between the weight of and the weight of nij 

remarked that 

Tension is always intermediate between the two... ^ 
Such a statement would be coded as a paraphrase. Another example would be the 
remark 

'There are no more forces' 
after reading the example statement 

Fa^ Fhy and Fc are all the forces acting on 
the body. 

A mathematical elaboration would be a statement such as 
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V/ this is the sine of 45 degrees, then this is the cosine 
of 4'^ degrees'. 

And & metastrateelc statement concerns what the students are dolnt or planning to do. 
such as: 

going to look at tht diagram ' 
VVn going to reread it' 
'This is something to remember'. 

* 

Table 2 shows the distribution of the three categories of statements for the Goo(*. 

» 

and Poor students. 

Looking at Table 2, one can see that although the Good students produced a 
significantly greater total number of elaborations (50.9) than the Poor students (17.5). 
they do not differ in the distribution of the proportion of each type of elaboration they 
provide. This similarity in the distribution suggests that the iJifference between the Good 
and Poor students may Just be a quantitative one. so that a simple remedial treatment 
for the Poor students Is simply to ask them to spend more time and effort at producing 
elaborations. However, closer examination of each of the categoriec suggests that the 
difference is not a straightforward quantitative one. Below, we analyze the differences in 
the structure and content of the Explanations, as well as differences in the > lonitoring 
Statements. 

Analyses of Explanations. 

Of all the elaboration statements produced by the Good students. zi% are 
explanations relating to the physics content as compared to 24% for the Poor students. 
Even though both groups produced proportionately a similar amount of explanations, the 
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Table 2 



Types of Elaborations 



GOOD POOR 

« 

Prop. Freq.* Prop. Freq.* 

Physics 

Explanation 31% 15.9 24% 4.3 

Monitoring 

Statements 39% 20.1 42% 7.3 

Others 
(Include 

Paraphrase) 30% 14.9 34% 5.9 



Total 100% 50.9 * 100% 17.5 



*An average number of statements per student per example. 

absolute number of .these explanations Is significantly different between the two groups 
(15.9 per example vs. 4.3, F(1.6) = 7.25. p <.05). Also, the correlation between the 
number of physics explanations and the subsequent success in solving the isomorphic 
problems Is very high (r = .76. p < .05). Thus it seems that these explanations play an 
Important role in learning from examples. 

Structure of the Explanations. One way to analyze the explanations students 
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provide upon studying example statements is to analyze the structure of .their 
explanations, by which we mean the form or purpose of what they say. Since our 
conjecture in this resea.rch was that students often fall to understand example solutions 
because the example provides neither a clear specification for why each procedural step 
is taken nor an explication of the consequences of each step, this would Imply that If 
students do understand an example solution ivell, they themselves must explicate the 
conditions and consequences of each solution step. Hence, we attempted to capture this 
cnaracterization in the choice of the analysis categories. Analyzing the physics 
explanation statements, we could classify them Into two major categories: justifications 
and diagrammatic statements (see Fig. 6 again). Diagrammatic statements are 
statements that basically describe In words what was depicted in the diagram. For 
example, a statement made by a student in reference to the diagram depicted In Figure l 
was: 

'Okay, so three forces are on the two strings 
and from the string going down to the object. ' 

Justifications on the other hand, are explanations that either: 

1) Refine or expand the conditions of an action. For example <n response 
to Line 18 of the Inclined plane example, which stated thi 

It is convenient to choose the x*ftxis of our reference f^ame 
to be along the incline and the y-axis to be normal to the 
incline 

student SI 10 explained the conditions of such a choice by saying: 

'and it is very, umm, wise to choose a reference frame that's 
parallel to the incline, parallel and normal to the incline, because 
that way, youV have to split up mg, the other forces are already, 
component vectors for you. ' 
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2) Explicate or Infer additional Implications of an action. In response 
to the same line In the inclined plane example, student SP2 would say: 

^So we can save 1 forct...wt save the F force. We save 
it 30, that I don 't have to calculate it by, with angles... 
So we can save two forces. Partitioning of two forces. ^ 

3) Impose a goal or purpose for an action^ In response 
to the same Line 18* again* SllO said: 

^Basically it looks like they are going to split up these three 
forces into their respective components. ' 

4) Give meaning to a set of quantitative expressions. In response to 
Lines 13-16 of Pig. 1, which simply states two equations, 
student SllO stated: 

^Umm, and looking along, they've done the same thing for FsubB. 
They've separated it into its components, into its component vectors, 
and tkey are basically able to figure out umm, 

the tension in each of the strings and how much W weights providing 
that...' 

Another student (SPl) responded similarly by saying: 

'Now they are going to do the same thing 
with it to the Y\ 



These latter two quotes show that not only can students understand in a sense the 
meaning and purpose of the quantitative expressions (which is a procedure for 
decomposing the forces), but they are also basically providing a goal for the set of 
actions. They refer to the goal of the equations as the "same thing*. 

Table 3 shows the distribution of the two kinds of Explanation statements. As can 
be seen* almost all of the Good students' explanations (98% or 15.5 statements) are of 
the Justification kind, whereas a fairly high proportion (35%) of the Poor students' 
explanations are diagrammatic statements. (The Interaction is significant at the .01 
level, P(l,6) = 20.55.) The characteristic of the diagrammatic statements Is that they do 
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not necessarily add any new information. In some sense, they are paraphrasing Into 
words what is shown pictorially, whereas the Justifications are explanations of various 
sorts as discussed above. Diagrammatic statements were -classified as "explanations" 
rather than Included In the "other" category since they were not paraphrases of textual 
statements. 

Tabb 3 

Kinds of Explanation 





GOOD 


POOR 

< 




Prop. 


Freq.* 


Prop. 


Freq.* 


Justifications 


9S% 


15.5 


65% 


2.75 


Diagrammatic 
Statements 


2% 


0.3 


35% 


1.5 


Total 


100% 


15.8 


100% 


4.25 



♦An average number of statements per student per example. 

Content of the Explanations. So far, we have Just given a characterization of the 
structure of the explanation statements. A majority of the explanations Justify, actions 
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stated In the text: they refine or expand the conditions of au action, explicate the 
consequences of an action, provide a goal for a set of actions, and explain the meaning: of 
a set of quantitative expressions. However, there is an important difference in the Good 
and Poor students' explan'^tions that is not easily captured by assessing the structure of 
what they say. This difference requires an analysis of the content of the explanations. 

We can first give a descriptive analysis of the difference between the Good and 
Poor students* Justifications by contrasting the following explanations generated after 
reading the equation from the Pulley example: 

T - ffijg = m^a 

A poor student (S109) comments: 

^OK, cctuse the acceleration ia due to gravity. ' 

This comment, at best, is an incomplete statement. It does not capture the interrelation 
between the force of tension and the force due to gravity. On the other hand, a Good 
student (SPl), in response to the same example line remarks: 

^OK, 30 its basically a way of adding them together and seeing if there is 
anything left over. And if there is anything left over, it equals the force: 
mass times acceleration. ' 

Such a comment Is not only more complete, but it shows that the Good student is trying 
to understand the example by relating the example statements to explanations and 
principles stated in the text, and understanding how one example statement follows from 
the previous one(s). This is typical of Good students* explanations. 
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Our descriptive analyses of the nature and content of the explanations so far 
indicate that the Good students seem to generate explanations which relate to the 
principles stated In the text, as well as relating the consecutive example statements to 
each other (as when they impose a goal on a set of actions, or give meaning to a set of 
quantitative expressions). We can further capture the degree to which students' 
explanations reflect the principles learned from the text by Judging the extent to which 
explanations can be said to be guided by these principles. For example, the following 
response to Line 3 of Figure 1: 

'5o that meana that they have to cancel out, only the body wonldnH he at 
teat. • 

would be judged to be guided by Newton's First Law that if there Is no motion, then the 
sum of the forces must be zero. Thus, to increase the rigor of our analyses, we further 
restricted our analyses to only those explanations that are derived or inferred from 
Newton's Three Laws. About a third of the Good students' explanations (4.75 out of 
15.8 statements) could be derived from or referred to Newton's Laws, whereas nearly 
none of the Poor students' explanations (0.7 out of 4.25 statements) related to any of 
Newton's Laws. 

One might think that more of the Good students' explanations aie derivable from 
the principles in the text than the Poor students' explanations because the Good students 
have understood the principles better prior to studying the examples. This interpretation 
can be ruled out by examining the content of the students' responses to the Declarative 
questions administered prior to the studying of the example-exercises. Recall that the 
students had to be able to answer a set of Declarative questions to the satisfaction of the 
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experimenters before they could proceed to the example-studying phase of the study. 
One part of this declarative test asks the students to state in their own words- Newton *s 
Three Laws. We analyzed those responses using the same analysis used in the Chi» Glaser 
and Rees (1982) Study Six. In this analysis each of the Newton's Laws was decomposed 
into several subcomponents. For example* Newton's Second Law, F=ma, has been 
decomposed into four subcomponents: 

• 

1. Applies to one body 

2. Involves all forces on the body 

3. Net force is the vector sum of all the forces 

4. F=ma, or the maesiitude of F is ma, and the direction of a is the same as F. 

Using these components as a scoring criterion, it can be seen in Table 4 that for the 
answers to the Declarative questions, both the Good and the Poor students encoded 5.5 
components (out of a total of 12 possible components for the three Laws). Thus, before 
studying the examples, the two groups do not differ in the amount of declarative 
knowledge that they have encoded. 

In order to compare the student's initial understanding of the Three Laws with their 
subsequent understanding as manifested in the explanations, we used the same 
component analysis to code those explanations which were previously Judged to be 
derivable from Newton's Three laws. Within these subsets of explanations, the Good 
students verbalized 5.5 components of the Three Laws whereas the Poor students only 
stated 1.25 components (t(B) = 2.14, p<.05). 
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Table 4 

Number of Components of Newton^s Three Laws (Out of 12) 



Declarative Test 
Explanations 
New Components 



GOOD POOR 

5*5 5*5 

5.5 1.25 

3.0 0.25 



Note*- The numbers represent averages per student. 

What is most remarkable Is that for the Good students, 3 out of the 5.5 
components stated in the explanations are distinct from those previously mentioned in 
the answers to Declarative questions. In other words, while example-studying, the Good 
students gained 3 additional components of the Three Laws. For example, three out of 
four Good students inferred or understood the second component, that the force 
"involves all the forces on the body.* This particular component of the Second Law was 
not mentioned initially by any of the Good students in the answers to the Declarative 
questions. However, self-explanations produced while studying the examples obviously 
instantiated this point. This suggests that the Good students can learn from the 
examples, perhaps because the examples instantiate components o{ the Laws that were 
not particularly salient from reading a declarative description of it in the text. Perhaps 
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-also, this Is why the Grood students feel so compelled to "explain" while studying, since 
they are learning and encodine new Icnowled^e. 

The Poor students, in contrast, hardly added any new components to what they 
already knew (0.25 new components, see Table 4). The difference in the number of new 
components added Is significantly different between the Good and the Poor students (t(d) 
= 3.67, p<.Ol). None of- the Poor students articulated, for example, the second 
component of Newton's Second Law, in either their explanations or their declarative 
answers. Thus, the Poor students did not add much new knowledge to what they 
already knew about the Three Laws, and further, did not use what they did know. 

Summary. Our analyses of explanations show that not only do the Good students 
produce a significantly greater number of explanations, but their explanations consist 
almost entirely of justifications which offer inferences about the conditions, the 
consequences, thie goals, and the meaning of various mathematical actions described in 
the example. On the other hand, over a third of the Poor students' explanations are 
diagrammatic statements which do not add any Inferences to what Is already described. 
Furthermore, the explanations that the Good students provide are largely guided by the 
principles, concepts, and definitions introduced in the text. 

We found that both the Good and the Poor students had basically the same 
understanding of these Three Laws initially, in terms of the number of components of 
the Three Laws that they understood, as cited in their answers to Declarative questions. 
(According to Greeno & Riley, 1987, explicit verbalization is the strictest criterion for 
assessing understanding of a principle). The self-explanations provided by the Good 
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Students while studying the examples made their understandlag of the principles more 
complete (8.5 out of 12 components). For example, after their self-explanations, the 
Good students understood 3 out of the 4 components of Newton's Second Law (the only 
component mlsslnt Is the first one), whereas prior to that, they only understood one of 
the four components (the last one). In contrast, since thft Poor students did not provide 
many explanations, they did not advance their understanding of the Laws after studying 
the examples (5.75 out of 12 components). For example, for the Second Law, no new 
component was added to what they already knew (which was only the fourth component 
of the Second Law). Hence, their understanding of the Second Law after studying the 
example was Just as Incomplete as It was before studying the example. In fact, they did 
not use as many components In their explanations as they could have from their Initial 
understanding. 

Since all the students seem to have encoaed the same number of components of the 
Three Laws Initially, we are assured that the students did acquire the same amount of 
relevant knowledge from the chapter. In particular, the Good students did not have a 
better understanding of Newton's Laws prior to studying the examples. However, we 
further propose that by the time students attempt to solve problems, their 
representations of the principles and other declarative knowledge Introduced in the text 
will differ depending on the degree to which their understanding of the principles is 
enhanced during their studying of examples. Such an interpretation Is consistent with 
the findings showing that students prefer text materials that contain examples (Reder et 
al., 1986; LeFevre & Dixon, 1986). Thus, even though both the Good and the Poor 
students can begin problem solving by applying general weak methods (the process of 
proceduralization, Anderson, 1987), the Good students will achieve greater success in 
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solvlne problems since an Instantiation of the weak methods depends on the 
representations of their declarative knowledge, which is* more complete. Thus** example* 
studying is a critical phase of skill acquisition. 

It is interesting to compare these findings to our expert*novice results* in which the 
experts cited many more components in their summaries of Newton's Three Laws than 
the novices (Chi, et.al., 1982). 'In contrast, before studying the examples, our Good and 
Poor students did not differ in their declarative understanding. Both groups have 
relatively poor understanding. As a result of differential learning from examples, the two 
groups developed knowledge differences which are compatible with differences we found 
between experts and novices. This suggests that experts became that way not becr»use 
they were better at encoding the declarative knowledge, but their greater understanding 
was probably gained from the way they studied examples and solved problems. 

While we .do n.ot understand the mechanism underlying self-explanations at this 
point, it Is clear that self-explanations not only construct better problem solving 
procedures, but they also help the students understand the underlying principles more 
completely. (More speculations about the mechanisms of self-explanations will be 
presented In the Discussion). 
Monitoring Statements 

So far, we have found that the Good students tend to explain an example to 
themselves (more so than the Poor students), and that these explanations are 
manifestations of learning that the Good students are undertaking while studying. The 
Poor students are not learning, at least not learning the components of the Three Laws, 
while studying the examples. Another possibility for why the Poor students are not 
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elaborating Is that they may not idealize that they do not understand the material: 
another possibility Is that they may actually think that they do understand the material, 
thus they need not elaborate. To see If these conjectures- are viable, we have examined 
the kind of elaborations that have been categorized as monitoring statements. As we 
noted earlier in Table 2, around 40% of the elaboration statements are of this kind (39% 
for the Good students and 42% for the Poor students). 

Monitoring statements can be further broken down Into those that iTidicate that 
the student understood what was presented In the example line (such as "Okay," or ■! 
can see now how they did it") and those that indicate that the student failed to 
understand (usually questions raised about the example line, such as 'Why is mgsinB^ 
negative?^. By examining the proportion of each type of monitoring statements, (see 
table 5), it seems that the Poor students detect comprehension failures less frequently 
than the Good students. Thus, the Poor students, within each example, generated an 
average of only i.i statements indicating that they failed to comprehend a line; whereas 
the Good students generated an average of 9.3 such coinments (t(6) = 2.32, p<.05). 
Common sense would have predicted that the Poor students should have detected 
comprehension failures more frequently, rather than less frequently, since they were less 
successful at solving problems than the Good students. 

Why Is it important to be able to detect comprehension failures? We surmise that 
it is important to be able to detect comprehensloa failures in order for students to know 
that they ought to do something to try and understand. One way to assess this Is to 
look at the frequency with which a detection of comprehension failure !s followed by 
explanations. Indeed, for both the Good and Poor students, detections of comprehension 
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Table 6 

Monitoring Statements 



Comprehension 
Failure 

Understand 



46% 9.3 
53% 10,8 



Total 



100% 20.1 



100% 



7.3 



*An average number of statements per student per example. 



failures do Initiate explanations, although more so for the Good than for the Poor 
students, both proportionately and on an absolute basis. Eighty-five percent (or 8.0 out 
of 9.3 statements) of the Good students' and 60% (or 0.6 out of l.i statements) oi the 
Poor students* detection of comprehension failure is followed by explanation, (F(2,12) = 
12.5, p < .01). These results suggest that one crucial advantage of the Good students is 
In their ability to spontaneously Identify the loci of their comprehension failures, which 
In turn Initiate the necessary Inferenclng process. 

Not only do the Poor students often fall to realize that they do not understand, but 
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on the few occasions that they do think they do not understand, these occur strictly at 
loci of quantitative expressions, whereas only half of the Good students* detection of 
comprehension failure arise from such loci. The other half reside in places where we 
believe important physics principles and concepts are belne explained. Our favorite 
example is the third line In Fie* 1» 

Consider the knot at the Junction of the three 
strings to be the body. 

At this location, 3 of the 4 Good students indicated comprehension failure, whereas none 
of the Poor students did. (This is a critical piece of information because it tells the 
student that the knot and not the mass of the block, should be the center of the 
reference frame at which all the forces have to sum to zero.) 

There Is also a qualitative difference in the kind of questions that detection of 
comprehension failure raises. For example, Poor students, when detecting their failure to 
comprehend, often state their lack of understanding in a general way , such as: 

^ell, what should you do here?' 

or restate the equation that they do not understand, such as: 
^Iwas having trouble with F-mg3in0=O^. 

On the other hand, the questions that the Good students raise are specific inquiries about 
the physical situation described in the example. For example, 

Vm wondering whether there would be acceleration due to gravity?' 
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38 
or 

^Why the force has to change?^ 

Thus» It appears that the specific questions that the Good students pose can be answered 
by encaeins in self-explanations, since the processes of self-explanations are essentially 
the processes of inferring the conditions and consequences of actions, inducing eoals, and 
so on. On the other hand, the general lack of understanding as posed by the questions of 
the Poor students cannot be resolved by engaging in self-explanations- 

In summary, the analyses of the monitoring statements essentially show that the 
Good students realize that they do not understand more often than the Poor students. 
The Poor students, in fact, seldom detect comprehension failures. When they do. It 
always occurs at loci of quantitative expressions. The ability to detect, comprehension 
failures Is important because such states (of incomprehension) tend to initiate 
explanation. When the Poor students do detect comprehension failures, tiielr sense of it 
is vague and general, whereas Good students ask very specific questions about what they 
don't understand. These specific questions can potentially be resolved by engaging in 
self-explanation. Id short, the Poor students seem oblivious to the fact that they do not 
understand, in part because they only have a superficial understanding of what they 
read. In the t»ext section, we will discuss how these different patterns of learning dictate 
how examples are used during problem solving. 

Problem Solving Protocols and References to Examples 
We now turn to our analyses of student's problem solving protocols with reference 
to the use of examples for the 12 isomorphic problems (4 per each of the 3 example 
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problems in the text). A global analysis is to examine the extent to which Good and Poor 
problem solvers use examples* References to examples were identined in three ways: (a) 
the student could make explicit remarks such as, "Now, Fm going to look at the 
example*"; (b) the student could be looking at the example, rereading some of the 
example lines; and (c) the student could respond that she/he is looking at the example if 
the experimenter probed and asked what she/he was doing* 

Both the Good and Poor students use examples frequently* The Good students 
referred to the examples in 0 out of the 12 isomorphic problems, and the Poor students 
referred to examples in solving 10 out of the 12 problems* Thus, at this global level, of 
whether or not students reference examples, our finding is consistent with those of Pirolll 
and Anderson (1985), as well as with the common intuition that students do refer to 
examples in learning to solve problems* However, a more detailed analysis shows that 
they reference examples in a very different way* 
Reference to Example Episodes 

At a more detailed level, we analyzed the number of "episodes* in which students 
refer to examples within each problem solving protocol* Only the first set of three 
Isomorphic problems (one per example) are used for this analysis* We analyzed only the 
first set because we were interested in the students* initial use and reliance on the 
studied example; subsequent problem solving could have benefited from using procedures 
that were learned while solving the first set* 

We can classify example usage into three categories: (a) Reading, (b) Copy and 
Map, and (c) Compare and Check* A Read episode is simply when students reread 
verbatim one or more example lines* A reading episode is marked by a reference to the 
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example, readine some consecutive lines (with or without coimnents Inserted), and 
terminating when the student eoes back to solving the problem. Copy and Map^ episodes 
are Instances where the student copies from the example either an equation, labels ftrom 
the diaeram, the tree body diagram itself, or the axes. The following quote for student 
S105 would be coded as mapping:: 

'OK, so choosing the axis from the diagram, it tvould be better to tilt it SO 
degrees. ' 

Compare and Check episodes are those In which the student turned to the example to 
check a specific subprocedure or a result, either before or after an Independent attempt 
on the solution. For example, one of the students (SPl) forgot what the units for weight 
were after resolving two vectors into four components and referred back to the example 
with the conmient 

Vn the example problem, how did they refer to weight?'. 

Table 6 shows the mean number of episodes of each kind for the problem solving 
protocols of the first set of Isomorphic problems. Although we showed earlier that at a 
global leveK both the Good and Poor students use examples equally often, this more 
detailed anjilysls shows that within a problem solving protocol, the Good students do 
refer less often to examples than the Poor students (2.7 episodes per problem vs. 6.8 
episodes per problem), although the difference Is only marginally significant (F(l,6) = 
3.46, p < .10). (It Is difficult to obtain dramatic differences statistically because the 
grain-size of this particular unit of analysis- the number of episodes-ls rather large.) 
The less frequent references to the example suggest that the Good students probably 
have extracted more oat of the examples while studying it (as evidenced by the greater 
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amount of elaboration), so that they may not need to refer to it as often during: problem 
solving. 

This interpretation is supported further by comparing the different kinds of 
reference to example episodes. The distribution of the different episodes is significantly 
different for the two groups (F(3,18) = 3.81, p < .05 for the interaction), predominantly 
because there is a substantial difference La the number of Reading epLtodes between the 
Good and Poor students (0.6 vs. 4.1, HSD p < .05, Tukey test). That Is, consistent with 
our interpretation, because the Poor students did not get much out of the examples when 
they were studying It, they now need to re-read them. (See Table 6). There are no 
differences in the number of Map and Check episodes. 

We further suggest that the Poor students need to reread the examples not only because 
they did not get much out of It while studying, but that they are using the examples to 
•find a solution*. This can be seen by the namber of lines that students read within 
each Reading episode. The Poor students read, on average, a larger number of example 
lines per episode than the Good students (13.0 vs. 1.6 lines, F(3,18) = ii.i, p < .oi). 
This difference also suggests that the function of the rereading for the Good and Poor 
students is different. The Good students seem to reread one line In the example as a way 
to locate certain relevant Information they need In order to check and compare their 
solutions, whereas the Poor students reread several lines until they encounter an 
equation that they can map. Once they come to the relevant Information, the Good and 
Poor students use the same number of lines for mapping (1.3 and 1.5 lines respectively) 
and for checking (0.5 and 0.3 respectively). 

44 



42 

Table 6 

Use of Example dnring Problem Solving 



No. of Episodes 
Per Problem 



GOOD 



POOR 



Read 

Map 

Check 



0.6 



1.1 



1.0 



4.1 



2.1 



0.2 



Total 



2.7 



6.6 



Within Each Read 
Episode: 



No. of Lines 
Read: 1.6 



Loci of 
Entry: 



Equation 
or FBD 



13.0 



first line 



Note. The numbers represent averages per student per problem. 

The fact that Good students reread a specific line in the example while solving a 
problem suggests that they consult the examples after they have a pian or formulated an 
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idea on how to solve the problem* whereas the Poor students use the examples as a way 
to find a solution that can be copied. Another way to support this conjecture Is to look 
at the location at which students begin reading the example lines. All the Poor students, 
for their first encounters with problem solving (during the first Isomorphic problem), 
started rereading the example from the very first llnc-r whereas none of the Good 
students started reading from the beginning. The Good students* first Interactions with 
an example usually consisted of referring to an equation or. to a free*body diagram. 

The contrast in our Interpretation can also be substantiated by the goals the 
students state whUe referring to the examples. Good students usually enter examples 
with a very specific goal. One student (SlOl), for example, said while referring to an 
example line: 

VVn looking at the formula here, trying to see how you solve for one (Force 
1) given the angle ' 

whereas Poor student refer to examples with a general global goal, such as: 

I 

What do they do?^ 
then proceed to read the first line. 

Learning From an Example Daring Problem Solving 

Since the Poor students spend a considerable amount of effort re-reading the 
examples while solving problems, it may be that some students prefer to learn in this 
context, while others prefer to study without a specific problem solving goal. To 
evaluate this conjecture, we looked for explanations generated during reference to 
example episodes within the problem solving protocols of the first set of isomorphic 
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problems for each of the three examples. Table 7 shows the number of each kind of 

explanations, using the same taxonomy we used earlier (see Table 3). 
Table 7 . . 

Frequency of Explanations Generated While 
Referring to Examples in the Context of 
Problem solving 



GOOD * POOR 

Justification 2.0 2.5 

Dia^ammatic 

Statement 0.25 0.25 



Total - 2.25 2.75 



Note, The numbers represent averages per student per problem. 



Only one Good student provided explanations. 



First of all. even though the Good students had as many explanations as the Poor 
students, (2.25 vs. 2.75), all the explanations of the Good students were generated by one 
student (Si 10) who had generated the least amount of explanations while studying 
examples. Many of the explanations provided by <;he Good students (as shown in Table 
3) were generated by only three out of the four Good students. Hence, this particular 
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Good student (Si 10) does fit our conjecture that some students prefer to explain while 
studying and others prefer to explain while solving problems* However, the amount of 
explanations provided by the Poor students in the problem solving context Is nowhere 
near the level that the Good students generated while studying examples (Compare 
Table 7 with Table 3)* If we combine the results of the explanation data generateu 
during problem solving together with those generated during studying, (i.e., Tables 3 and 
7), the resulting table will have exactly the same pattern as Table 3. 

Diacussion 

Our research queried the extent to which Individual difference in learning to solve 
problems is attributable to difference in the way knowledge Is encoded from the 
exampie-exerclses. We found, in general, that Good students (those who have greater 
success at solving problems) tend to study example-exercises in a text by explaining and 
providing justifications for each action (i.e., t^eir explanations refine and expand the 
conditions of an action, explicate the consequences of an action, provide a goal for a set 
of actions, relate the consequences of one action to anothei\ and explain the meaning of a 
set of quantitative expressions). More importantly, their explanations relate the actions 
to the principles and concepts in the text, which in turn further enhance their 
understanding of the principles. Essentially, Good students read the example with 
understanding. Poor students do not often explain the example-exercises to themselves. 
When they do, their explanations do not seem to connect with their understanding of the 
principles and concepts in the text. 

Good students can also accurately monitor their comprehension failures and 
successes, while studying examples. This Is consistent with what is generally found in 
the literature (see Brown, Bransford, Ferrara & Campione, 1983). They seem to detect 
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accurately when they do and do not understand* Such accuracy in detection Is extremely 
Important because It leads to an attempt to try and - understand the material* Poor 
students, on the other hand, seem less accurate at detecting comprehension failures/ 
When they do. these oc^nr where mathematical expressions are being manipulated, 
rather than at places where conceptual principles are being instantiated* 

Finally, we found that the Good students used the examples in a very different way 
from the Poor students* In general. Good students, during problem solving, used the 
examples for a specific reference, whereas Poor students reread them as If to search for a 
solution* This Interpretation Is consistent with the following set of behaviors* First, 
Good students refer to the examples less frequently than the Poor students, within each 
solution attempt* Second, when they do refer to the examples. Good students read only 
one or two lines, whereas Poor students read around 13 lines* The Good students read 
only one or two Unps because they have a specific goal In mind* Once they found what 
they needed from the examples, both the Good and Poor students can auequately map 
the example and check It for accuracy* These differences In the ways Good and Poor 
students use examples suggest that the Good (but net the Poor) students have 
understood the examples when they studied It, as a consequence of the explanations they 
generated, and now can use the examples as a reference* 

Our results stand In contrast to the mixed results often obtained In research 
examining the role of elaboration In problem solving* It jas sometimes been found that 
elaboration generally do not facilitate problem solving (Reder et al 1986; Reed et al*, 
1985)* The difference between these studies and the present one Is that we examine the 
elaborations and explanations students spontaneously produced, whereas these other 
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studies provided the elaborations for the students to read. The difficulty of interpreting 
results of a study which provides the elaboration is that when it fails to facilitate 
problem solving, one cannot know whether the elaborations themselves are inadequate, 
or whether providing elaborations is futile. 

Although the task in which we observed learning was how students studied 
example-exercises, we believe that our results generalize to how good and poo* students 
learn many kinds of materials, such as declarative text (Bransford, Stein, Vye, Pranks, 
Auble, Mezynski, & Perfetto, 1982), diagrams, tables, charts, and so on, as well as 
solving other types of problems* In fact, ouf results are reminiscent of an earlier study by 
Gagne and Smith (1962) which showed that asking students to verbalize while they solve 
a 3-disc puzzle-type problcr i correlated *-lgnificantIy with success at solving problems. 
Although Gagne and Smith did not report individual differences, they were able to 
categorize the protocols ?^to 4 types: 

1) those which orier..2d toward single moves in the solution of the problems, v/ith 
explanations such as: 

'only possible move ' 
''Just to try it' 
'don't know' 

2) those which anticipated to the extent of two moves, with comments such as: 

'To get at the larger disc' 
'to free up a space ' 

3) reasons which anticipated sequences of moves, such as: 
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'move as with a tkrtt^ise sequence ' 

'if disc is odd-numberedy move to circle B' 

4) and reasons which explained the prlnciplest such as: 

'move odd-numhered disc in the clockwise direction ' 

'move even^numbered discs in the counterclockwise direction '. 

In light of our present analysis, we would expect the poor solvers not only to produce 
fewer explanations (i.e^ less articulation), but also to produce explanations that are not 
necessarily Justifications (as those In Type 1). We would expect good solvers to produce 
both more explanations and also to produce explanations which basically state subeoals» 
(as those in Type 2), which induce a goal from, a sequence of actions (as those in Type 3), 
and which relate actions to principles (as those in Type 4)* Unfortunately, Gagne and 
Smith did not break their students down into good and poor solvers* In other words, we 
do not think that articulating an explanation per se is the critical factor (as suggested by 
Gagne and Smith); rather, what the students articulated is the most /important factor. 

Why do self-explanations help understanding and problem solving? To put it in 
another way, what Is learned from self-explanations? We have several tentative 
conjectures, which may not be mutually exclusive. One interpretation is that self- 
explanations consist of the creation of inference rules that are instantiations of the 
principles and definitions introduced in the text. These inference rules are specific to the 
example presented. (They may be generalized later during episodes of learning from 
problem solving.) For example, we can collapse related explanations into inference rules. 
They are related usually if they refer to the same conditions. For example, the last 5 
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explanations shown In Fig. 5 (HI - Vn)» can be converted to three Inference rules (shown 
In Pie. 7). • 

Rule 1: IP there is a body, and It has weight 

THEN the weight will act as a force. 

Rule 2: IP there are forces upholding a body 

THEN they are resistant to the body's weight and 
they are equal to he orce of weight. 

Rule 3: IF (there is the force of the weight of the body 

and there are resistance forces...) 
THEN they will all equal out. 

Figure 7: Coding of lines 6-13 of Figure 5 Into three Inference rules. 

These Inference rules may be taken as Instantiations of the definition of a force (Rule 1), 
application of Newton's Third Law (Rule 2), and instantiation of Newton's Second Law 
(Rule 3). In fact, Larkin and Simon (1987) found it necessary to model problem solving 
by the use of similar kinds of inference rules. The present study provides evidence to 
support such a claim. In tact, our data further support Larkin and Simon's (1987) 
conjecture that 'students may well be unable to solve problems in part because they 
learn prlncipbr^. and do not translate them Into inference rules' (p. 75). As roted earlier, 
we found no difrerences in the abilities of the Good and Poor students to write down the 
declarative definitions and principles Introduced in the text part of the chapter. 
However, unless students translate the declarative knowledge into specific procedural 
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inference rules, they cannot use them to solve problems. 

Another interpretation of self-explanations, based on the analyses of the- 
explanations which related to Newton's Laws, is the idea that students, while reading the 
text, have only encoded a subset of the components of the Laws. That is, some features 
of the Laws are not noticed as Important, until one sees that they are emphasized in the 
example. Through self*explanations, the students are relating: these new components of 
the Laws to components they already know. 

A third way to look at the role of self-explanations Is to say that the declarative 

Laws have been encoded entirely, but that some components are more accessible than 

others, and thus more easily articulated explicitly (as in the case of* answering: 

Declarative questions about them). On the other hand, self*expIanations during studying 

examples may make other latent or implicit components more accessible. Our data 

cannot discriminate between the states of knowledge in which this information was 

available but inaccessible versus not available at all. ' 

• 

Another possible mechanism underlying self-explanations may be that self- 
explanations produce a qualitative constraint network which represents knowledge of the 
solution steps. For instance, a system of trigonometry constraints could link together a 
general equation, such as + F^^^ = 0, the diagram of forces, and a specific equation. 

o o 

-F^cosSO r Fj^cos45 = 0. When a similar problem is encountered, say one with 
different angles, qualitative propagation through the constraint network can yield a plan 
for the quantitative solution. More importantly, tho constraints can propagate 
information in both forwards and backwards directions, so that the above constraint 
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system could be used to eenerate a plan ;'or solving for one anele given the values of the 
other anele and the net force* Such qualitative constraint networks may be the kind of 
inference structure generated by self-explanations, at least Initially (VanLehn, personal 
communication)* 

Finally, models do exist in the AI literature which can generalize from a single 
training example (Lewis, 1086; Mitchell, Keller & Kodar-Cabelli, 1086)* Such 
generalizations are acquired by constructing a proof (explanation) of how an example is 
an instance of the concept which it exemplifii^s, t^^en, within the forrnal theory of the 
task domain, the explanation is generalized to cover a larger class of examples* Cur data, 
at a gross level, provide empirical evidence to suggest th^t such explanation-based 
learning may indeed be the way students generalize from single examples. Howe* er, we 
need a more detailed look at th^se models to see the extent to which they can 
accommodate our data* In Lewis' (1086) model (EXPL), the system basically contains a 
set of heuristics for detecting the relation between the action taken and the outcome of 
an action* For example, in learning text editing, suppose a student is observing a 
demonstration of the following action and its outcome: the screen draws a box around 
an object, and shortly after, the object in the box disappears* An IDENTITY heuristic 
will be able to conjecture that the relation between the action of drawing a box around 
an object causes the object to disappear* Thus, the function of explanations is to apply 
these heuristics to come up with a hypothesis of the relation between the action and the 
outcome* Notice that these heuristics aro domain-indepen^ t. I;\ our data, 
understanding an action and Its outcome requires conjecturing not only a relation 
between the action and Its outcome, but also to explain how that action-outcome pair Is 
an instantiation of a principle which Is part of the domain theory. Thus, the greatest 
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difference between what our students can do and what EXPL can do is that EXPL 
cannot capture the rcle of the domain knowledge. In fact, the to-be-learned domain 
(text-editlng) is very syntactic In nature, and does not require domain knowledge In 
order to understand the example actions. 

In Mitchell et al.'s (1986) system (EES), however, domain knowledge is provided. 
The contrast between Mitchell'^ theory and our data is that Mitchell's system has "too 
much" domain knowledge in the following sense: when an example has to be proved to 
be an Instance of a goal concept, the system already has a clear and a complete 
definition of this goal concept. Thus, the function of explanation is to produce a proof 
that the example is an Instance of the goal concept. This may require that some features 
of the example be Inferred so that the features of the example will match those de/ined 
in the goal concept. If we assume that the knowledge that permits these Inferences to be 
made on the example concept Is analogous to the domain knowledge of physics, then this 
particular aspect of the model does seem to simulate parts of our students* behavior. 
However, the key difference between the knowledge possessed by the EES system and 
our students Is that EES has all the knowledge of the goal concept, whereas our students 
do not. For example, our students do not have a complete understanding of Newton's 
Laws. Furthermore, the function of students' explanations seems to be both the 
enhancement of their initial understanding and generation of new understanding. 

In sum, while our data does seem to support these two AI models In a general way, 
detailed analyses point out that these models are Incomplete In describing actual 
learning, mainly because students' Initial learning from text is more limited than the 
EES model, for example, and yet their subsequent learning from the examples Is more 
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sophisticated. 

We conclude by once again quoting Simon (1979), with regard to^ the 
characteristics of the learner: 

It is left largely to the student (by examining worke<5-out examples, by working 
problems, or in some other way) to generate the productions, the condltion- 
' action pairs, that are required to solve problems (p. 92, emphasis added). 

We believe that the "some other way* that students learn is via generating and 
completing explanations. Hence, we have provided evidence for Simon's, insight that 
•students learn both by being taught and by self-instrnction • (p. 87), whereby self- 
instruction is mediated by self-explanations. 
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Notes 

^Newton's First Law states that "Every body persists in Its state of rest or of 
uniform motion in a straight line unless it is compelled to change that state by forces 
Impressed on it. 
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